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Editorial 


The Peer-Review Process and 
Its Relationship With Environmental 
and Occupational Health 


Peer review of journal articles is now the norm in virtual- 
ly all scientific disciplines.! The art of peer review has ad- 
vanced over the years, and scientific publishing has em- 
braced new thinking. In this editorial, we explore the basics 
of peer review and options for improving the process. 

Winston Churchill once said democracy is the “worst 
form of government except all those other forms that have 
been tried.” Much the same could be said of peer review, in 
that it is the worst way to assess the value of work—except 
for all the alternatives. Authors worry about careless as- 
signment of manuscripts, technical competence or bias 
among reviewers, and having their ideas stolen. Although 
all these horrors are possible, in a well-managed journal 
they seldom occur. Managing peer review effectively re- 
quires vigilance and close supervision, and this effort is 
usually rewarded by added value because papers are not 
only graded and sorted but are also better integrated into 
the scientific literature because of the input of thoughtful 
reviewers. 

Cautious deliberation and evaluation is at the heart of the 
scientific process.” Peer review is the scientific community 
assessing the worth and meaning of a body of work before 
it is published. It is the first step in assimilating findings or 
insights into the worldview of scientific thought and inte- 
gration into the broader scientific literature. Peer review is 
not only a quality-assurance mechanism, but also a means 
by which significant findings or insights are better integrat- 
ed into the broader sphere of scientific knowledge. This oc- 
curs when reviewers identify hitherto unreported connec- 
tions and, ideally, make constructive connections that link 
the current work more firmly to existing bodies of knowl- 
edge. Reviewers also assess the worth of a new submission 
in relation to existing knowledge in the field, all in the in- 
terest of enabling a more effective presentation to the sci- 
entific community at large. At the least, a conscientious re- 
viewer (1) judges whether the presented findings accord 
with the predominant paradigm of scientific thinking at the 


Spring 2008, Vol. 63, No. 1 


time and (2) draws the author’s attention to various anom- 
alies that may need explaining. 

In general, there are 3 main objectives of the peer-review 
process: (1) to prevent the publication of bad material, (2) to 
improve journal scholarship, and (3) to improve a manu- 
script’s language and data presentation.* Any practicing 
editor is also aware however, that peer review is not and can- 
not be absolute. An excellent paper may be unsuitable for a 
particular journal. A paper that is acceptable but not monu- 
mental may still advance knowledge in a certain way or 
meet a particular need for data. There always will be bad re- 
viewers of good papers, as well as split decisions to be man- 
aged. Groundbreaking work is seldom well-documented and 
irrefutable when it first appears, but neither is junk science. 
Editors are responsible for discerning the difference in such 
cases, albeit with guidance from reviewers. Regardless of 
any scientific article’s intrinsic quality, the publication 
process also involves a certain amount of politics.* At the 
least, it requires a delicate web of interaction and sometimes 
negotiation among editors and reviewers on the basis of 
shared values, knowledge, and assumptions about the field. 

Similarly, the decision process entails a series of compli- 
cated interactions between many participants. A previous 
editorial in the New England Journal of Medicine, for ex- 
ample,> suggested that more than 20 physicians and scien- 
tists are involved in the decision-making process of each 
manuscript they receive. Although this is probably an ex- 
treme, most biomedical journals now follow a similar model 
when dealing with new submissions, whereby the first step 
involves an initial review by the editor in chief (or managing 
editor) to assess the suitability and appropriateness of its 
general content. Editors consider themes and features of a 
paper that are important to the mission of the journal and 
those which they believe will be important to readers.® 
Sometimes the journal’s mission is also the editor’s person- 
al mission, but this close identification can also lead to 
clouded judgment and bias. For such reasons, there should 
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always be some governance check on an editor’s authority, 
an active editorial board, and (at least passive) oversight by 
the publisher. Governance itself is a topic for another edito- 
rial, but suffice it to say that it represents one of the less ap- 
preciated dimensions in the art of scientific publication 
management. 

After passing a front-gate process, the editor usually as- 
signs manuscripts to an executive or associate editor with 
experience in the particular field, who then organizes at least 
2 peer reviewers to assess it.’ After receiving their appraisal 
of the manuscript, the associate editor reviews the manu- 
script again and makes an initial decision to accept, revise, 
or reject the article. Aside from manuscripts that are imme- 
diately rejected, most authors are required to revise their pa- 
pers and relatively few are accepted outright.® If the review- 
ers’ opinions are discordant, the editor seeks a tiebreaking 
third and sometimes fourth reviewer, followed by an editor- 
ial meeting to decide the manuscript’s ultimate fate. In the 
interest of time, the tie-breaking reviewer may be consulted 
informally or may be an in-house reader—such as another 
editor—but the tie-breaking reviewer should always have 
sufficient expertise in the field to be as qualified as their ini- 
tial reviewing counterpart. 

Although the decision to accept or reject is fairly clear, 
inviting authors to revise and resubmit their manuscripts 
conveys some ambiguity. It does not guarantee acceptance. 
Rather, it allows authors another opportunity to present their 
work more effectively for their scientific peers and in a for- 
mat more consistent with the aims of the journal. A decision 
for major revision implies that an author’s work does not fit 
content or meet standards within a particular field and thus 
needs to be reshaped to clarify the general message or to 
match the standards of a particular discipline. Minor revi- 
sion implies that the work has a clear message but could be 
more effectively presented or should undergo correction of 
insubstantial errors. Despite this seemingly objective 
process, authors, scientists and scholars have often regarded 
the concept of peer review as a somewhat dubious and mys- 
terious process. Regardless of the system used, authors are 
bound to fret about their submissions—Campion et al,’ for 
example, compared a manuscript out for peer review to a 
child away at camp. 

In any case, there are 3 main types of peer review used for 
the assessment of scientific manuscripts: single blinded, double 
blinded, and fully open. 


Single-Blinded Peer Review 


A single-blinded peer-review system reveals authors’ 
names to the reviewer, but reviewers themselves remain 
anonymous to the author. The majority of general medical 
journals used this method until recently, mainly because of 
custom.” The advantage of a single-blinded peer-review 
process is that it puts an author’s submission in the context 
of their previous work and thereby gives a reviewer clues, by 
reputation and knowledge of the institution or group, to the 
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credibility of the current work.!? However, the process is 
deeply flawed due to the competitive aspects of human na- 
ture. The same clues that lend credibility may also yield se- 
vere disadvantages for new investigators, maverick col- 
leagues, and emerging institutions. Revealing the authors’ 
names and institutions to reviewers also permits the 
Matthew Effect,’ whereby manuscripts submitted by well- 
known people tend to be accepted, regardless of quality. The 
primary criticism of single-blinded peer review is the per- 
ceived lack of transparency because important judgments 
made about others’ work should not be done in secret.” 
Authors may feel defenseless against the arbitrary behavior 
of referees who cannot be held accountable.'° Transparency 
virtually demands accountability because reviewers will be- 
come known as biased or arbitrary if they do not behave.” 
When transparency is absent from the review process, au- 
thors may perceive it as secretive and suspicious.!! Further- 
more, a lack of transparency allows reviewers to hold up 
their competition, punish authors they do not like, or settle 
old scores.* Even so, by the late 20th century, many well- 
known journals were still using such a system.!” 


Double-Blinded Peer Review 


Because of substantial drawbacks in the single-blinded 
system, there are really only 2 acceptable versions of peer 
review: completely closed or completely open.'? A double- 
blinded peer-review process is completely closed, a system 
in which neither the author nor the reviewer knows the 
other’s identity. Editors also may take steps to remove po- 
tential identifiers from the methods section and the refer- 
ence list, to further ensure anonymity. A commonly cited 
disadvantage of the double-blinded system is that, ultimate- 
ly, masking the identity of all authors—particularly high- 
profile scientists—is difficult.'* Some studies also indicate 
that the success of blinding does not differ much between 
journals that have such a policy and those that do not!* and 
that masking the author’s identity may not necessarily im- 
prove the quality of reviews. 


Fully Open Peer Review 


Fully open peer review has been gaining popularity in re- 
cent years, as shortcomings of the aforementioned methods 
have become more widely known and possibly also because 
editors in chief have increasingly sought the author’s input 
on how best to manage the review process for their particu- 
lar journal. A fully open system is cheap and easy to man- 
age, and any journal can readily implement it.!? Nonethe- 
less, it is not a panacea. A completely open peer-review 
system may suffer, for example, if junior reviewers are re- 
luctant to criticize the work of senior researchers for fear of 
reprisals,? which may lead to a growing imbalance in politi- 
cal power.* However, as in the single-blinded system, man- 
uscripts submitted by famous people may be automatically 
accepted, regardless of quality, because of intimidation or 
awe.’ Whether practicable or not, this system is clearly 
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unpopular with reviewers: in a recent study, one-third of re- 
viewers said they wished to remain anonymous." 


Author-Nominated Reviewers 


A counter to the potential for bias or lack of expertise in 
editor-assigned reviewers is a system that allows the author 
to nominate reviewers whom they believe have sufficient in- 
sight and expertise to judge their work fairly. Potential for 
bias in the other direction is, of course, obvious because 
these nominees may not be objective or may be colleagues 
of the author whose views are already known. Although 
author-suggested reviewers tend to make more favorable 
recommendations for acceptance of an article, the quality of 
their review is believed to be similar to reviewers who are 
editor-selected.'® 


Peer Review and Occupational Health Journals 


Occupational and environmental health is a classic 
boundary discipline,'’ or Grenzgebiete, as these fields were 
a century ago called in German. Many journals in this field 
evolved from the medical publication model. A recent in- 
vestigation of the peer-review process used in dedicated oc- 
cupational health periodicals found that although the vast 
majority of international publications now use a double- 
blinded peer-review system, some continued to use the sin- 
gle-blinded process.'* However, of those with a predomi- 
nately single-blinded system, some articles or issues were 
also reviewed in a double-blinded manner.'? For this rea- 
son, some experts have recently proposed that manuscripts 
submitted to occupational health journals be subject to uni- 
form requirements.'* Although such calls have lagged far 
behind the general medical community, which first sug- 
gested uniform requirements for biomedical journals in 
1978,° the time is right for an increasing degree of stan- 
dardization and accountability within the publication 
processes of our discipline. 


General Criticisms and the Way Forward 


A continuing and fundamental debate regarding peer re- 
view is whether it actually improves the quality of published 
research. Einstein’s groundbreaking Annus Mirabilis Papers 
were not peer reviewed, nor was Watson and Crick’s 1951 
article on DNA structure.*! One reason is that most early 
medical journals, such as the Lancet, followed the model of 
journalism and frequently mixed opinion with data presen- 
tation. Other early journals began as the proceedings of a 
learned society, with articles intended to be comprehensive 
and accurate accounts of oral presentations or correspon- 
dence rather than archives of discrete written reports. Since 
the mid-17th century when they began to be published, sci- 
entific journal administrators often have received criticism 
regarding what they print.”* Structured peer reviewing did 
not become widespread until after the second world war.” 

By 1965, Science** had described peer reviewers’ essen- 
tial responsibilities, although it noted that the system would 
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work only if referees understood what they were supposed 
to do. An intrinsic criticism the entire peer-review process, 
then as now, regards the suitability of reviewers chosen by 
the editor and whether their expertise is valid.! The presti- 
gious Proceedings of the National Academy of Sciences 
does not practice peer review, on the basis of the theory that 
members of the Academy already have demonstrated scien- 
tific excellence and are therefore capable of deciding the 
merits of their own work. Even so, scientists tend to write 
articles for different reasons than they read them,” and there 
is increasing pressure in biomedical science to have one’s 
work published in the leading journals.* Journal prestige is 
also a factor,”° and editors may assign prestigious, although 
potentially inappropriate, reviewers. 

Publication bias toward positive results—perceived or 
otherwise—has always troubled potential authors, but neg- 
ative results do get published.*° Manuscripts initially re- 
jected by a top journal tend to be published in smaller peri- 
odicals,”’ and at least half all published occupational 
medicine articles will eventually be listed in Medline.”® 
Even so, the decision to publish one’s important findings in 
specialist occupational medicine journals rather than their 
more general medical counterparts creates a certain struc- 
tural disadvantage for the author, not the least of which is 
being published in a journal with a relatively low impact 
factor.”? Articles published in such journals also receive less 
coverage, given the fact that as journals specialize, they 
tend to become relevant to fewer people.” 

The time has come for environmental and occupational 
health journals to converge on standard procedures, for- 
mats, and benchmarks of quality. But environmental and 
occupational health is not biochemistry, nor is it a medical 
specialty. Its methods and approaches vary with the prob- 
lem at hand, and the subject matter is heterogeneous. For 
this field, standardization should not lie in the uniformity in 
technique but, rather, in a more consistent level of quality 
across whatever discipline, method, or approach applies to 
the problem under study. Consistency in quality and a 
process that supports these standards, whatever the applica- 
tion, is much needed. 

For all its faults, the peer-review process is still the best 
strategy for assessing manuscripts fairly and can result in 
the substantial improvement of papers prior to publica- 
tion.!! To facilitate this, occupational health journals must 
begin adopting standardized peer-review guidelines and 
editorial policies.!® Achieving editorial consistency and re- 
viewer accountability is clearly a step in the right direction 
toward improving the quality of all manuscripts in the 
field. 
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Double-Blinded Peer Review 
for Manuscripts Submitted 
to the Archives of Environmental 
& Occupational Health 


Beginning January 2008, the Archives of Environmental & Occupational Health moved to a com- 
pletely double-blinded peer-review process. This means that neither the author’s nor the reviewer’s 
identity will be disclosed to either party. As such, there are a few changes that authors must heed for 
the new system to be effective. Making these changes prior to submission will help authors reduce 
delays in manuscript review. 

Each submission should now include at least 2 files: (1) a cover page containing the title of the 
article, what type of article it is (ie, full-length manuscript, brief communication, or case study), 
and the authors’ names, affiliations, and address for correspondence; and (2) the main document, 
whose first page should contain only the title, abstract, and keywords. The text of the manuscript 
should begin on the second page of this file, and any accompanying tables or figures should be up- 
loaded as separate files. 


Page One 
¢ Title of the article 
¢ Type of article 
¢ Author name(s) and affiliation(s) 


Page Two 
¢ Title of the article 
¢ Structured abstract (maximum 135 words) 
* Keywords 


In the article text, authors should avoid referring to their institution by name, as well as any other 
statements that might make them potentially identifiable to reviewers. Authors should also avoid 
(where possible) citing their previously published articles by name in the text. 

We trust that the adoption of a double-blinded system for the Archives will improve the quality 
of the review process, reduce bias, and signify a progressive move toward standardization among 
manuscripts submitted to journals of occupational and environmental health. 
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